Personalized item recommendations through large-scale deep-embedding architecture with real-time inferencing

ABSTRACT

A method including training two sets of item embeddings for items in an item catalog and a set of user embeddings for users, using a triple embeddings model, with triplets. The triplets each can include a respective first user of the users, a respective first item from the item catalog, and a respective second item from the item catalog, in which the respective first user selected the respective first item and the respective second item in a respective same basket. The method also can include generating an approximate nearest neighbor index for the two sets of item embeddings. The method additionally can include receiving a basket including basket items selected by a user from the item catalog. The method further can include grouping the basket items of the basket into categories based on a respective item category of each of the basket items. The method additionally can include randomly sampling a respective anchor item from each of the categories. The method further can include generating a respective list of complementary items for the respective anchor item for the each of the categories based on a respective lookup call to the approximate nearest neighbor index using a query vector associated with the user and the respective anchor item. The method additionally can include building a list of personalized recommended items for the user based on the respective lists of the complementary items for the categories. The method further can include sending instructions to display, to the user on a user interface of a user device, at least a portion of the list of personalized recommended items. Other embodiments are disclosed.

TECHNICAL FIELD

This disclosure relates generally to providing personalizedrecommendations through large-scale deep-embedding architecture.

BACKGROUND

Item recommendations can assist a user when selecting items online.Online grocery shopping can be different from general merchandise onlineshopping, as grocery shopping is often highly personal, users often showboth regularity in purchase types and purchase frequency, and userstypically exhibit specific preferences for product characteristics, suchas brand affinity for milk or price sensitivity for wine.

BRIEF DESCRIPTION OF THE DRAWINGS

To facilitate further description of the embodiments, the followingdrawings are provided in which:

FIG. 1 illustrates a front elevational view of a computer system that issuitable for implementing an embodiment of the system disclosed in FIG.3;

FIG. 2 illustrates a representative block diagram of an example of theelements included in the circuit boards inside a chassis of the computersystem of FIG. 1;

FIG. 3 illustrates a block diagram of a system that can be employed forproviding personalized recommendations through large-scaledeep-embedding architecture, according to an embodiment;

FIG. 4 illustrates a block diagram showing a triple embeddings modelused to represent users, items, and baskets, based on a skip-gramframework;

FIG. 5 illustrates a flow chart for a method, according to anembodiment;

FIG. 6 illustrates a flow chart for a method, according to anotherembodiment;

FIG. 7 illustrates a block diagram of a system that can be employed forproviding personalized recommendations through large-scaledeep-embedding architecture, according to another embodiment;

FIG. 8 illustrates a graph showing inference latency (in milliseconds(ms)) versus basket size (in number of basket items); and

FIG. 9 illustrates a flow chart for a method, according to anotherembodiment.

For simplicity and clarity of illustration, the drawing figuresillustrate the general manner of construction, and descriptions anddetails of well-known features and techniques may be omitted to avoidunnecessarily obscuring the present disclosure. Additionally, elementsin the drawing figures are not necessarily drawn to scale. For example,the dimensions of some of the elements in the figures may be exaggeratedrelative to other elements to help improve understanding of embodimentsof the present disclosure. The same reference numerals in differentfigures denote the same elements.

The terms “first,” “second,” “third,” “fourth,” and the like in thedescription and in the claims, if any, are used for distinguishingbetween similar elements and not necessarily for describing a particularsequential or chronological order. It is to be understood that the termsso used are interchangeable under appropriate circumstances such thatthe embodiments described herein are, for example, capable of operationin sequences other than those illustrated or otherwise described herein.Furthermore, the terms “include,” and “have,” and any variationsthereof, are intended to cover a non-exclusive inclusion, such that aprocess, method, system, article, device, or apparatus that comprises alist of elements is not necessarily limited to those elements, but mayinclude other elements not expressly listed or inherent to such process,method, system, article, device, or apparatus.

The terms “left,” “right,” “front,” “back,” “top,” “bottom,” “over,”“under,” and the like in the description and in the claims, if any, areused for descriptive purposes and not necessarily for describingpermanent relative positions. It is to be understood that the terms soused are interchangeable under appropriate circumstances such that theembodiments of the apparatus, methods, and/or articles of manufacturedescribed herein are, for example, capable of operation in otherorientations than those illustrated or otherwise described herein.

The terms “couple,” “coupled,” “couples,” “coupling,” and the likeshould be broadly understood and refer to connecting two or moreelements mechanically and/or otherwise. Two or more electrical elementsmay be electrically coupled together, but not be mechanically orotherwise coupled together. Coupling may be for any length of time,e.g., permanent or semi-permanent or only for an instant. “Electricalcoupling” and the like should be broadly understood and includeelectrical coupling of all types. The absence of the word “removably,”“removable,” and the like near the word “coupled,” and the like does notmean that the coupling, etc. in question is or is not removable.

As defined herein, two or more elements are “integral” if they arecomprised of the same piece of material. As defined herein, two or moreelements are “non-integral” if each is comprised of a different piece ofmaterial.

As defined herein, “approximately” can, in some embodiments, mean withinplus or minus ten percent of the stated value. In other embodiments,“approximately” can mean within plus or minus five percent of the statedvalue. In further embodiments, “approximately” can mean within plus orminus three percent of the stated value. In yet other embodiments,“approximately” can mean within plus or minus one percent of the statedvalue.

As defined herein, “real-time” can, in some embodiments, be defined withrespect to operations carried out as soon as practically possible uponoccurrence of a triggering event. A triggering event can include receiptof data necessary to execute a task or to otherwise process information.Because of delays inherent in transmission and/or in computing speeds,the term “real-time” encompasses operations that occur in “near”real-time or somewhat delayed from a triggering event. In a number ofembodiments, “real-time” can mean real-time less a time delay forprocessing (e.g., determining) and/or transmitting data. The particulartime delay can vary depending on the type and/or amount of the data, theprocessing speeds of the hardware, the transmission capability of thecommunication hardware, the transmission distance, etc. However, in manyembodiments, the time delay can be less than approximately 0.5 second,one second, or two seconds.

Description of Examples of Embodiments

Turning to the drawings, FIG. 1 illustrates an exemplary embodiment of acomputer system 100, all of which or a portion of which can be suitablefor (i) implementing part or all of one or more embodiments of thetechniques, methods, and systems and/or (ii) implementing and/oroperating part or all of one or more embodiments of the non-transitorycomputer readable media described herein. As an example, a different orseparate one of computer system 100 (and its internal components, or oneor more elements of computer system 100) can be suitable forimplementing part or all of the techniques described herein. Computersystem 100 can comprise chassis 102 containing one or more circuitboards (not shown), a Universal Serial Bus (USB) port 112, a CompactDisc Read-Only Memory (CD-ROM) and/or Digital Video Disc (DVD) drive116, and a hard drive 114. A representative block diagram of theelements included on the circuit boards inside chassis 102 is shown inFIG. 2. A central processing unit (CPU) 210 in FIG. 2 is coupled to asystem bus 214 in FIG. 2. In various embodiments, the architecture ofCPU 210 can be compliant with any of a variety of commerciallydistributed architecture families.

Continuing with FIG. 2, system bus 214 also is coupled to memory storageunit 208 that includes both read only memory (ROM) and random accessmemory (RAM). Non-volatile portions of memory storage unit 208 or theROM can be encoded with a boot code sequence suitable for restoringcomputer system 100 (FIG. 1) to a functional state after a system reset.In addition, memory storage unit 208 can include microcode such as aBasic Input-Output System (BIOS). In some examples, the one or morememory storage units of the various embodiments disclosed herein caninclude memory storage unit 208, a USB-equipped electronic device (e.g.,an external memory storage unit (not shown) coupled to universal serialbus (USB) port 112 (FIGS. 1-2)), hard drive 114 (FIGS. 1-2), and/orCD-ROM, DVD, Blu-Ray, or other suitable media, such as media configuredto be used in CD-ROM and/or DVD drive 116 (FIGS. 1-2). Non-volatile ornon-transitory memory storage unit(s) refer to the portions of thememory storage units(s) that are non-volatile memory and not atransitory signal. In the same or different examples, the one or morememory storage units of the various embodiments disclosed herein caninclude an operating system, which can be a software program thatmanages the hardware and software resources of a computer and/or acomputer network. The operating system can perform basic tasks such as,for example, controlling and allocating memory, prioritizing theprocessing of instructions, controlling input and output devices,facilitating networking, and managing files. Exemplary operating systemscan include one or more of the following: (i) Microsoft® Windows®operating system (OS) by Microsoft Corp. of Redmond, Wash., UnitedStates of America, (ii) Mac® OS X by Apple Inc. of Cupertino, Calif.,United States of America, (iii) UNIX® OS, and (iv) Linux® OS. Furtherexemplary operating systems can comprise one of the following: (i) theiOS® operating system by Apple Inc. of Cupertino, Calif., United Statesof America, (ii) the Blackberry® operating system by Research In Motion(RIM) of Waterloo, Ontario, Canada, (iii) the WebOS operating system byLG Electronics of Seoul, South Korea, (iv) the Android™ operating systemdeveloped by Google, of Mountain View, Calif., United States of America,(v) the Windows Mobile™ operating system by Microsoft Corp. of Redmond,Wash., United States of America, or (vi) the Symbian™ operating systemby Accenture PLC of Dublin, Ireland.

As used herein, “processor” and/or “processing module” means any type ofcomputational circuit, such as but not limited to a microprocessor, amicrocontroller, a controller, a complex instruction set computing(CISC) microprocessor, a reduced instruction set computing (RISC)microprocessor, a very long instruction word (VLIW) microprocessor, agraphics processor, a digital signal processor, or any other type ofprocessor or processing circuit capable of performing the desiredfunctions. In some examples, the one or more processors of the variousembodiments disclosed herein can comprise CPU 210.

In the depicted embodiment of FIG. 2, various I/O devices such as a diskcontroller 204, a graphics adapter 224, a video controller 202, akeyboard adapter 226, a mouse adapter 206, a network adapter 220, andother I/O devices 222 can be coupled to system bus 214. Keyboard adapter226 and mouse adapter 206 are coupled to a keyboard 104 (FIGS. 1-2) anda mouse 110 (FIGS. 1-2), respectively, of computer system 100 (FIG. 1).While graphics adapter 224 and video controller 202 are indicated asdistinct units in FIG. 2, video controller 202 can be integrated intographics adapter 224, or vice versa in other embodiments. Videocontroller 202 is suitable for refreshing a monitor 106 (FIGS. 1-2) todisplay images on a screen 108 (FIG. 1) of computer system 100 (FIG. 1).Disk controller 204 can control hard drive 114 (FIGS. 1-2), USB port 112(FIGS. 1-2), and CD-ROM and/or DVD drive 116 (FIGS. 1-2). In otherembodiments, distinct units can be used to control each of these devicesseparately.

In some embodiments, network adapter 220 can comprise and/or beimplemented as a WNIC (wireless network interface controller) card (notshown) plugged or coupled to an expansion port (not shown) in computersystem 100 (FIG. 1). In other embodiments, the WNIC card can be awireless network card built into computer system 100 (FIG. 1). Awireless network adapter can be built into computer system 100 (FIG. 1)by having wireless communication capabilities integrated into themotherboard chipset (not shown), or implemented via one or morededicated wireless communication chips (not shown), connected through aPCI (peripheral component interconnector) or a PCI express bus ofcomputer system 100 (FIG. 1) or USB port 112 (FIGS. 1-2). In otherembodiments, network adapter 220 can comprise and/or be implemented as awired network interface controller card (not shown).

Although many other components of computer system 100 (FIG. 1) are notshown, such components and their interconnection are well known to thoseof ordinary skill in the art. Accordingly, further details concerningthe construction and composition of computer system 100 (FIG. 1) and thecircuit boards inside chassis 102 (FIG. 1) are not discussed herein.

When computer system 100 in FIG. 1 is running, program instructionsstored on a USB drive in USB port 112, on a CD-ROM or DVD in CD-ROMand/or DVD drive 116, on hard drive 114, or in memory storage unit 208(FIG. 2) are executed by CPU 210 (FIG. 2). A portion of the programinstructions, stored on these devices, can be suitable for carrying outall or at least part of the techniques described herein. In variousembodiments, computer system 100 can be reprogrammed with one or moremodules, system, applications, and/or databases, such as those describedherein, to convert a general purpose computer to a special purposecomputer. For purposes of illustration, programs and other executableprogram components are shown herein as discrete systems, although it isunderstood that such programs and components may reside at various timesin different storage components of computing device 100, and can beexecuted by CPU 210. Alternatively, or in addition to, the systems andprocedures described herein can be implemented in hardware, or acombination of hardware, software, and/or firmware. For example, one ormore application specific integrated circuits (ASICs) can be programmedto carry out one or more of the systems and procedures described herein.For example, one or more of the programs and/or executable programcomponents described herein can be implemented in one or more ASICs.

Although computer system 100 is illustrated as a desktop computer inFIG. 1, there can be examples where computer system 100 may take adifferent form factor while still having functional elements similar tothose described for computer system 100. In some embodiments, computersystem 100 may comprise a single computer, a single server, or a clusteror collection of computers or servers, or a cloud of computers orservers. Typically, a cluster or collection of servers can be used whenthe demand on computer system 100 exceeds the reasonable capability of asingle server or computer. In certain embodiments, computer system 100may comprise a portable computer, such as a laptop computer. In certainother embodiments, computer system 100 may comprise a mobile device,such as a smartphone. In certain additional embodiments, computer system100 may comprise an embedded system.

Turning ahead in the drawings, FIG. 3 illustrates a block diagram of asystem 300 that can be employed for providing personalizedrecommendations through large-scale deep-embedding architecture,according to an embodiment. System 300 is merely exemplary andembodiments of the system are not limited to the embodiments presentedherein. The system can be employed in many different embodiments orexamples not specifically depicted or described herein. In someembodiments, certain elements, modules, or systems of system 300 canperform various procedures, processes, and/or activities. In otherembodiments, the procedures, processes, and/or activities can beperformed by other suitable elements, modules, or systems of system 300.In some embodiments, system 300 can include a personalizedrecommendation system 310 and/or web server 320.

Generally, therefore, system 300 can be implemented with hardware and/orsoftware, as described herein. In some embodiments, part or all of thehardware and/or software can be conventional, while in these or otherembodiments, part or all of the hardware and/or software can becustomized (e.g., optimized) for implementing part or all of thefunctionality of system 300 described herein.

Personalized recommendation system 310 and/or web server 320 can each bea computer system, such as computer system 100 (FIG. 1), as describedabove, and can each be a single computer, a single server, or a clusteror collection of computers or servers, or a cloud of computers orservers. In another embodiment, a single computer system can hostpersonalized recommendation system 310 and/or web server 320. Additionaldetails regarding personalized recommendation system 310 and/or webserver 320 are described herein.

In some embodiments, web server 320 can be in data communication throughInternet 330 with one or more user devices, such as a user device 340.User device 340 can be part of system 300 or external to system 300. Insome embodiments, user device 340 can be used by users, such as a user350. In many embodiments, web server 320 can host one or more websitesand/or mobile application servers. For example, web server 320 can hosta website, or provide a server that interfaces with a mobileapplication, on user device 340, which can allow users to browse and/orsearch for items (e.g., products), to add items to an electronic cart,and/or to purchase items, in addition to other suitable activities.

In some embodiments, an internal network that is not open to the publiccan be used for communications between personalized recommendationsystem 310 and web server 320 within system 300. Accordingly, in someembodiments, personalized recommendation system 310 (and/or the softwareused by such systems) can refer to a back end of system 300 operated byan operator and/or administrator of system 300, and web server 320(and/or the software used by such systems) can refer to a front end ofsystem 300, and can be accessed and/or used by one or more users, suchas user 350, using user device 340. In these or other embodiments, theoperator and/or administrator of system 300 can manage system 300, theprocessor(s) of system 300, and/or the memory storage unit(s) of system300 using the input device(s) and/or display device(s) of system 300.

In certain embodiments, the user devices (e.g., user device 340) can bedesktop computers, laptop computers, a mobile device, and/or otherendpoint devices used by one or more users (e.g., user 350). A mobiledevice can refer to a portable electronic device (e.g., an electronicdevice easily conveyable by hand by a person of average size) with thecapability to present audio and/or visual data (e.g., text, images,videos, music, etc.). For example, a mobile device can include at leastone of a digital media player, a cellular telephone (e.g., asmartphone), a personal digital assistant, a handheld digital computerdevice (e.g., a tablet personal computer device), a laptop computerdevice (e.g., a notebook computer device, a netbook computer device), awearable user computer device, or another portable computer device withthe capability to present audio and/or visual data (e.g., images,videos, music, etc.). Thus, in many examples, a mobile device caninclude a volume and/or weight sufficiently small as to permit themobile device to be easily conveyable by hand. For examples, in someembodiments, a mobile device can occupy a volume of less than or equalto approximately 1790 cubic centimeters, 2434 cubic centimeters, 2876cubic centimeters, 4056 cubic centimeters, and/or 5752 cubiccentimeters. Further, in these embodiments, a mobile device can weighless than or equal to 15.6 Newtons, 17.8 Newtons, 22.3 Newtons, 31.2Newtons, and/or 44.5 Newtons.

Exemplary mobile devices can include (i) an iPod®, iPhone®, iTouch®,iPad®, MacBook® or similar product by Apple Inc. of Cupertino, Calif.,United States of America, (ii) a Blackberry® or similar product byResearch in Motion (RIM) of Waterloo, Ontario, Canada, (iii) a Lumia® orsimilar product by the Nokia Corporation of Keilaniemi, Espoo, Finland,and/or (iv) a Galaxy™ or similar product by the Samsung Group of SamsungTown, Seoul, South Korea. Further, in the same or different embodiments,a mobile device can include an electronic device configured to implementone or more of (i) the iPhone® operating system by Apple Inc. ofCupertino, Calif., United States of America, (ii) the Blackberry®operating system by Research In Motion (RIM) of Waterloo, Ontario,Canada, (iii) the Android™ operating system developed by the OpenHandset Alliance, or (iv) the Windows Mobile™ operating system byMicrosoft Corp. of Redmond, Wash., United States of America.

In many embodiments, personalized recommendation system 310 and/or webserver 320 can each include one or more input devices (e.g., one or morekeyboards, one or more keypads, one or more pointing devices such as acomputer mouse or computer mice, one or more touchscreen displays, amicrophone, etc.), and/or can each comprise one or more display devices(e.g., one or more monitors, one or more touch screen displays,projectors, etc.). In these or other embodiments, one or more of theinput device(s) can be similar or identical to keyboard 104 (FIG. 1)and/or a mouse 110 (FIG. 1). Further, one or more of the displaydevice(s) can be similar or identical to monitor 106 (FIG. 1) and/orscreen 108 (FIG. 1). The input device(s) and the display device(s) canbe coupled to personalized recommendation system 310 and/or web server320 in a wired manner and/or a wireless manner, and the coupling can bedirect and/or indirect, as well as locally and/or remotely. As anexample of an indirect manner (which may or may not also be a remotemanner), a keyboard-video-mouse (KVM) switch can be used to couple theinput device(s) and the display device(s) to the processor(s) and/or thememory storage unit(s). In some embodiments, the KVM switch also can bepart of personalized recommendation system 310 and/or web server 320. Ina similar manner, the processors and/or the non-transitorycomputer-readable media can be local and/or remote to each other.

Meanwhile, in many embodiments, personalized recommendation system 310and/or web server 320 also can be configured to communicate with one ormore databases, such as a database system 317. The one or more databasescan include a product database that contains information about products,items, or SKUs (stock keeping units), for example, among otherinformation, as described below in further detail. The one or moredatabases can be stored on one or more memory storage units (e.g.,non-transitory computer readable media), which can be similar oridentical to the one or more memory storage units (e.g., non-transitorycomputer readable media) described above with respect to computer system100 (FIG. 1). Also, in some embodiments, for any particular database ofthe one or more databases, that particular database can be stored on asingle memory storage unit or the contents of that particular databasecan be spread across multiple ones of the memory storage units storingthe one or more databases, depending on the size of the particulardatabase and/or the storage capacity of the memory storage units.

The one or more databases can each include a structured (e.g., indexed)collection of data and can be managed by any suitable databasemanagement systems configured to define, create, query, organize,update, and manage database(s). Exemplary database management systemscan include MySQL (Structured Query Language) Database, PostgreSQLDatabase, Microsoft SQL Server Database, Oracle Database, SAP (Systems,Applications, & Products) Database, and IBM DB2 Database.

Meanwhile, personalized recommendation system 310, web server 320,and/or the one or more databases can be implemented using any suitablemanner of wired and/or wireless communication. Accordingly, system 300can include any software and/or hardware components configured toimplement the wired and/or wireless communication. Further, the wiredand/or wireless communication can be implemented using any one or anycombination of wired and/or wireless communication network topologies(e.g., ring, line, tree, bus, mesh, star, daisy chain, hybrid, etc.)and/or protocols (e.g., personal area network (PAN) protocol(s), localarea network (LAN) protocol(s), wide area network (WAN) protocol(s),cellular network protocol(s), powerline network protocol(s), etc.).Exemplary PAN protocol(s) can include Bluetooth, Zigbee, WirelessUniversal Serial Bus (USB), Z-Wave, etc.; exemplary LAN and/or WANprotocol(s) can include Institute of Electrical and Electronic Engineers(IEEE) 802.3 (also known as Ethernet), IEEE 802.11 (also known as WiFi),etc.; and exemplary wireless cellular network protocol(s) can includeGlobal System for Mobile Communications (GSM), General Packet RadioService (GPRS), Code Division Multiple Access (CDMA), Evolution-DataOptimized (EV-DO), Enhanced Data Rates for GSM Evolution (EDGE),Universal Mobile Telecommunications System (UMTS), Digital EnhancedCordless Telecommunications (DECT), Digital AMPS (IS-136/Time DivisionMultiple Access (TDMA)), Integrated Digital Enhanced Network (iDEN),Evolved High-Speed Packet Access (HSPA+), Long-Term Evolution (LTE),WiMAX, etc. The specific communication software and/or hardwareimplemented can depend on the network topologies and/or protocolsimplemented, and vice versa. In many embodiments, exemplarycommunication hardware can include wired communication hardwareincluding, for example, one or more data buses, such as, for example,universal serial bus(es), one or more networking cables, such as, forexample, coaxial cable(s), optical fiber cable(s), and/or twisted paircable(s), any other suitable data cable, etc. Further exemplarycommunication hardware can include wireless communication hardwareincluding, for example, one or more radio transceivers, one or moreinfrared transceivers, etc. Additional exemplary communication hardwarecan include one or more networking components (e.g.,modulator-demodulator components, gateway components, etc.).

In many embodiments, personalized recommendation system 310 can includea communication system 311, an item-to-item system 312, a basket-to-itemsystem 313, a triple embeddings system 314, a post-processing system315, an approximate nearest neighbor (ANN) index system 316, and/ordatabase system 317. In many embodiments, the systems of personalizedrecommendation system 310 can be modules of computing instructions(e.g., software modules) stored at non-transitory computer readablemedia that operate on one or more processors. In other embodiments, thesystems of personalized recommendation system 310 can be implemented inhardware. Personalized recommendation system 310 and/or web server 320each can be a computer system, such as computer system 100 (FIG. 1), asdescribed above, and can be a single computer, a single server, or acluster or collection of computers or servers, or a cloud of computersor servers. In another embodiment, a single computer system can hostpersonalized recommendation system 310 and/or web server 320. Additionaldetails regarding personalized recommendation system 310 and thecomponents thereof are described herein.

In many embodiments, system 300 can provide item recommendations to auser (e.g., as customer) based on items that the user has included in abasket of selected items. These recommended items can be selected by theuser to supplement the basket of the user. These item recommendationscan be personalized to the user, based on personal preferences of theuser. With growing consumer adoption of online grocery shopping throughplatforms such as Amazon Fresh, Instacart, and Walmart Grocery, it canbe advantageous to provide relevant recommendations at one of morepoints of the shopping experience. Online grocery shopping is typicallyhighly personal. Users often show both regularity in purchase types andpurchase frequency, as well as exhibit specific preferences for productcharacteristics, such as brand affinity for milk or price sensitivityfor wine.

In a number of embodiments, system 300 can provide a within-basketrecommender, which can suggest grocery items that go well with the itemsin a shopping basket (e.g., cart) of the user, such as milk withcereals, or pasta with pasta sauce. In practice, users often purchasegroceries with a particular intent, such as for preparing a recipe orstocking up for daily necessities. In several embodiments, thewithin-basket recommender can consider both (i) item-to-itemcompatibility within a shopping basket and (ii) user-to-item affinity,which together can advantageously generate complementary and relevantproduct recommendations that are actually user-personalized.

In many embodiments, a triple embeddings model can be trained and usedfor generating personalized recommendations. The triple embeddings modelcan be similar or identical to the triple2vec model described inMengting Wan et al., “Representing and recommending shopping basketswith complementarity, compatibility and loyalty,” in Proceedings of the27th ACM International Conference on Information and KnowledgeManagement, ACM (Association of Computing Machinery), 2018, pp.1133-1142, which is incorporated by reference herein in its entirety.

Turning ahead in the drawings, FIG. 4 illustrates a block diagram 400showing a triple embeddings model used to represent users, items, andbaskets, based on a skip-gram framework, as described in Wan et al.,supra. Item representation learning approaches based on a skip-gramframework generally seek to find item representations that are usefulfor predicting contextual (e.g., related) items or users, by definingdifferent “context windows.” These context windows can be implemented invarious different instantiations on a heterogeneous graph, with nodesthat represent items, users, or baskets.

As shown in FIG. 4, block diagrams 400 can include graphs 410, 420, and430, each of which includes a user node for a user u₁, an item node foran item i₃, an item node for an item i₄, and a basket node for a basketb₁₂. Item-basket links represent the item being in the basket, such asitem i₃ and item i₄ both being in b₁₂. User-basket links represent theuser having selected (e.g., purchased) the basket, such as user u₁having purchased basket b₁₂. The triple embeddings model thus usestriplets of (user, first item, second item), indicating that the firstand second items were bought by the user in the same basket. The tripleembeddings model can be trained, such that, given any two of theelements of the triplet, the third element of the triplet can bepredicted. For example, graph 410 shows that user u₁ can be predictedgiven item i₃ and item i₄, graph 420 shows that item i₄ can be predictedgiven user u₁ and item i₃, and graph 430 shows that item i₃ can bepredicted given user u₁ and item i₄.

In a number of embodiments, the triple embeddings model can be trainedusing past purchase data for users to derive embeddings that representthe users and the items from the triplets. For example, the tripleembeddings model learns an embedding vector h_(u) for the user u and adual set of embedding vectors (p_(i), q_(j)) for the item pair (i, j).These embeddings can be modeled by taking a dot product between each ofthe embedding vectors, such that a cohesion score s_(i,j,u) for atriplet can be defined as follows:

s _(i,j,u) =p _(i) ^(T) q _(j) +p _(i) ^(T) h _(u) +q _(j) ^(T) h _(u)

where T represents the transpose of the vector. The cohesion score cancapture both user-item compatibility through p_(i) ^(T)h_(u) and q_(j)^(T)h_(u), and item-item complementarity through p_(i) ^(T)q_(j).

In several embodiments, the embeddings can be learned by maximizing theco-occurrence log-likelihood L (loss function) of each triplet, which isdefined as follows:

$L = {\sum\limits_{{({i,j,u})} \in T}\left( {{\log{P\left( {\left. i \middle| j \right.,\ u} \right)}} + {\log\;{P\left( {\left. j \middle| i \right.,\ u} \right)}} + {\log{P\left( {\left. u \middle| i \right.,j} \right)}}} \right)}$

where T is the set of triplets, P(i|j, u) is the softmax probability,defined as follows:

${P\left( {\left. i \middle| j \right.,\ u} \right)} = \frac{e^{S_{i,j,u}}}{\sum_{i^{\prime}}e^{S_{i^{\prime},j,u}}}$

and where P(j|i, u) and P(u|i, j) can be similarly obtained byinterchanging (i,j) and (i, u), respectively. In many embodiments, thesoftmax function can be approximated by a Noise Contrastive Estimation(NCE) loss function, in accordance with many skip-gram models withnegative sampling. A log-uniform (e.g., Zipf) distribution can be usedto sample negative examples.

As an example, the triple embeddings model can be trained for the twosets of item embeddings (p, q) and the user embeddings (h) by randomlyinitializing a 128 dimension vector for each of these embeddings from auniform distribution of [−0.01, 0.01]. After initialization, the tripleembeddings models can be trained with an adaptive moment estimationoptimizer, such as Adam, which is a variation of stochastic gradientdescent (SGD), as follows:

$\left. m_{w}^{({t + 1})}\leftarrow{{\beta_{1}m_{w}^{(t)}} + {\left( {1 - \beta_{1}} \right){\nabla_{w}L^{(t)}}}} \right.\left. v_{w}^{({t + 1})}\leftarrow{{\beta_{2}v_{w}^{(t)}} + {\left( {1 - \beta_{2}} \right)\left( {\nabla_{w}L^{(t)}} \right)^{2}}} \right.{{\overset{\hat{}}{m}}_{w} = \frac{m_{w}^{({t + 1})}}{1 - \left( \beta_{1} \right)^{l + 1}}}{{\overset{\hat{}}{v}}_{w} = \frac{v_{w}^{({t + 1})}}{1 - \left( \beta_{2} \right)^{t + 1}}}\left. w^{({t + 1})}\leftarrow{w^{(t)} - {\eta\frac{{\overset{\hat{}}{m}}_{w}}{\sqrt{{\overset{\hat{}}{v}}_{w}} + \epsilon}}} \right.$

where w^((t)) are the parameters of the model, is the co-occurrencelog-likelihood loss function described above, β₁ and β₂ are forgettingfactors for the gradients and second moments of the gradients, η is alearning rate, and t is a time step. As an example, the tripleembeddings model can be trained end-to-end for 100 epochs using 500million triplets, using a past purchase data set over a one year timeframe with 800 million user-item interactions, with 3.5 million users,and 90 thousand items, in which frequency threshold-based user level anditem level filters were used to remove cold start users and items fromthe training.

Once the triple embeddings model is trained, matrix P and matrix Q canstore the two sets of trained item embeddings for the catalog of items,such that matrices P and Q are each real-valued matrices having a numberof rows equal to the number of items in the item catalog (which can belimited to not include cold start items, as described above) and anumber of columns equal to the dimension of the embedding vector, suchas 128, as described above. Matrix H can store the trained userembeddings for the users, such that matrix H is a real-valued matrixhaving a number of rows equal to the number of users and a number ofcolumns equal to the dimension of the embedding vector, such as 128, asdescribed above.

For a given “anchor” item j and a given user u, as inputs, the traineditem matrices P and Q and trained user matrix H can be used to computethe cohesion score for each of the items i, to determine a score thatindicates how complementary item i is to anchor item j for user u. Forexample, consider a simplified 32-dimension example in which the traineduser embedding vector h_(u) in matrix H for user u is as follows:

[0.32738936, 0.47486708, −0.44018468, −1.0137466, 0.4358432,−0.21896638, −1.1607007, 0.3042493, 0.48715204, 0.7864144, −0.8422068,0.2846775, −1.0895154, 0.40973258, −0.13478273, 0.38279486, −0.56316096,−0.6631576, 0.4856452, 0.06135664, −0.49751332, −0.42942294, −0.6039675,−0.9324385, −0.01547593, 0.959067, 0.5826312, −0.1542099, −0.2449495,−0.6153408, −0.05710425, 0.4830378];the trained item embedding vector p_(i) in matrix P for item i is asfollows:

[−0.3057416 0.07774266 0 803581 0.72900647 0.09258661 0.027785880.2705059 −0.07434104 −0 08514664 −0.33368888 0.25841185 0.5306720.23466173 0.17158407 0.4228771 −0.6867255 0.24365486 0.3469819−0.20994641 0.4170019 −0 17914794 0.13667138 −0.02210519 0.373138640.40211987 −0.2390854 0 4869946 0.34046495 −0.568138 0.5706644−0.11231046 −0.2386148];and the trained item embedding vector q_(j) in matrix Q for anchor itemj is as follows:

[−0.48197877 −0.47216713 0.7691373 0.31753355 −0.34225956 0.072890840.71213585 −0.19382164 0.2557742 −0.6527458 0.07196312 0.029753660.06880591 −0.38648534 0.3167588 0.16804916 0.4228312 0.38767454−0.86445206 0.03228619 −0.31139407 0.40430707 −0.41132057 0.20389684−0.18048657 −0.91251445 0.13429038 0.595438 −0.02878908 0.4773644−0.27105173 0.14990729].The cohesion score can be calculated, as follows:

s _(i,j,u) =p _(i) ^(T) q _(j) +p _(i) ^(T) h _(u) +q _(j) ^(T) h_(u)=−5.4485517.

In many embodiments, for the anchor item j and user u, given as inputs,the top k complementary items (i) can be determined as outputs byiterating through the items (i) in matrix P and computing the cohesionscore, and selecting the top k items (i). This approach can be describedas follows:

$\underset{i}{argmax}\left( {{p_{i}^{T}q_{j}} + {p_{i}^{T}h_{u}} + {q_{j}^{T}h_{u}}} \right)$

for the top k items, which k can be set as a design parameter. As anexample, k can be set to 15 to determine the top 15 complementary items(i), given anchor item j and user u. The triple embeddings model canadvantageously be used to recommend one or more items (i) that arepersonalized for a user, when the user (u) has selected an anchor item(j), which can be referred to as an item-to-item model.

In several embodiments, the item-to-item model additionally can includea complementary category filtering technique, which can filter out itemsthat are recommended due to being popular overall items. For example, inonline grocery shopping, bananas, milk, eggs, and bath tissue are verypopular items. These items would often be included as recommendationsfrom the item-to-item model, merely due to their popularity in mostcarts, despite not being particularly complementary to a given anchoritem j, such as specific type of dry pasta, for a particular user u.

In many embodiments, the complementary category filtering technique canbe based on subcategories that are complementary to the subcategory ofthe anchor item. In a number of embodiments, each item in the itemcatalog can include an item taxonomy, which can include at least thefollowing four levels: Level 1 (L1) for Super Department, Level 2 (L2)for Department, Level 3 (L3) for Category, and Level 4 (L4) forSub-category. For example, a specific item having item title “GreatValue Vitamin D Whole Milk,” can have L1 of “Eggs & Dairy,” L2 of“Milk”, L3 of “Dairy Milk,” and L4 of “Whole Milk.” The complementarycategory filtering technique can be performed at the L4 subcategorylevel, by considering other subcategories that are complementary to thesubcategory of the anchor item, and boosting the scores for items inthose subcategories. In several embodiments, the complementary categoryfiltering technique can involve calculating support and lift metrics, asfollows:

${{Support}(A)} = {{{fraction}\mspace{14mu}{of}\mspace{14mu}{all}\mspace{14mu}{tractions}\mspace{14mu}{that}\mspace{14mu}{contain}\mspace{14mu}{the}\mspace{14mu}{item}\mspace{14mu}{A.{{Lift}\left( A\rightarrow B \right)}}} = \frac{{Support}\left( {A\bigcup B} \right)}{{Support}\;(A) \times {{Support}(B)}}}$

where A is a given anchor item, and B is an item to be recommended fromanchor item A, denoted as (A→B). When B is popular item, but unrelatedto anchor item A, the lift metric will be low. When B is complementaryto A, but not merely popularly co-bought, the lift metric will be high.

Using these lift scores, other subcategories that are complementary tothe subcategory of the anchor item can be determined, based on the liftmetrics for one or more of the items in the other subcategories. For agiven anchor item j, such as specific type of dry pasta, the item(L1/L2/L3/L4) taxonomy can be as follows: (Pantry/Pasta & Pizza/DryPasta/Dry Pastas). Using the lift scores, complementary subcategories,based on the top 10 lift scores, can be determined to be as follows:

-   -   1. Pantry/Pasta & Pizza/Pasta Sauce/Pasta & Pizza Sauces    -   2. Eggs & Dairy/Cheese/Shredded Cheese/Mozzarella Cheeses    -   3. Pantry/Soup/Broth, Stocks & Bouillon/Stocks, Broth    -   4. Pantry/Rice, Grains & Dried Beans/Rice/Rice Mixes    -   5. Pantry/Soup/Ready to Eat/Veggie Soups    -   6. Pantry/Soup/Ready to Eat/Meat & Seafood Soups    -   7. Pantry/Pasta & Pizza/Macaroni & Cheese/Pasta Mixes    -   8. Pantry/Rice, Grains & Dried Beans/Rice/Rice    -   9. Pantry/Soup/Ramen & Dry Soup Mix/Ramen & Udon    -   10. Meat/Beef/Ground Beef & Patties/Fresh

In many embodiments, the lift scores approach can determine items thatare complementary and related more accurately than using co-boughtscores. Using co-bought scores instead for the dry pasta anchor itemwould have found many popular but unrelated subcategories, such as Eggs,Bananas, Low-Fat Milk, Bath Tissue, and Whole Milk.

In some embodiments, the complementary category filtering technique caninvolve applying the lift scores to the complementary items, such thattruly complementary items are boosted more, while popular yet unrelateditems are boosted less, such that these latter items can drop lower inthe score ranking and be effectively filtered out.

In many embodiments, this item-to-item model can be used to recommendpersonalized items for user u when given an anchor item j, such that therecommended items are complementary to each other and not unrelatedpopular co-bought items. For example, for a specific anchor item havingitem title “Swiffer Sweeper Wet Mopping Cloths, Open-Window Fresh, 24count,” and having (L1/L2/L3) taxonomy of (Household Essentials/CleaningTools/Brooms, Mops & Brushes), the item-to-item model can recommend theitems shown below in Table 1:

TABLE 1 L1 - Super Item Title Department L2 - Department L3 - CategoryAngel Soft Toilet Household Essentials Paper Products Bath Tissue Paper,12 Jumbo Rolls Great Value Everyday Household Essentials Paper ProductsPaper Towels Strong Printed Paper Towels, 3 Big Rolls Clorox Clean-UpAll Household Essentials Cleaning Products All Purpose Cleaners PurposeCleaner with Bleach, Spray Bottle, Fresh Scent, 32 ounce (oz) DownyUltra Liquid Household Essentials Laundry Fabric Softeners FabricConditioner, April Fresh, 105 Loads 90 fluid oz all with StainliftersHousehold Essentials Laundry Detergents Free Clear Liquid LaundryDetergent, 123 Loads, 184.5 oz Great Value Ultra Household EssentialsPaper Products Bath Tissue Strong Paper Towels, Split Sheets, 6 DoubleRolls Great Value Everyday Household Essentials Paper Products PaperTowels Strong Printed Paper Towels, 8 Count Lysol All Purpose HouseholdEssentials Cleaning Products All Purpose Cleaners Cleaner Spray, LemonBreeze, 32 oz Bounce Dryer Sheets, Household Essentials Laundry FabricSofteners Outdoor Fresh, 80 Count Gain Aroma Boost Household EssentialsLaundry Detergents Liquid Laundry Detergent, Original

With a single item selected by the user, the item-to-item model thus canprovide a list of complementary item recommendations that arepersonalized to the user. Often, especially in the online groceryshopping context, baskets often have more than one item, and often havemany items. In several embodiments, a basket-to-item model can be usedto provide personalized item recommendations, given a basket of itemsselected by a user. In many embodiments, the basket-to-item model canuse the item-to-item model described above, as described below infurther detail.

Turning ahead in the drawings, FIG. 5 illustrates a flow chart for amethod 500, according to an embodiment. In some embodiments, method 500can be a method of providing personalized item recommendations from abasket of items for a user, which can be referred to as a basket-to-itemmodel. Method 500 is merely exemplary and is not limited to theembodiments presented herein. Method 500 can be employed in manydifferent embodiments or examples not specifically depicted or describedherein. In some embodiments, the procedures, the processes, and/or theactivities of method 500 can be performed in the order presented. Inother embodiments, the procedures, the processes, and/or the activitiesof method 500 can be performed in any suitable order. In still otherembodiments, one or more of the procedures, the processes, and/or theactivities of method 500 can be combined or skipped.

In many embodiments, system 300 (FIG. 3), personalized recommendationsystem 310 (FIG. 3), and/or web server 320 (FIG. 3) can be suitable toperform method 500 and/or one or more of the activities of method 500.In these or other embodiments, one or more of the activities of method500 can be implemented as one or more computing instructions configuredto run at one or more processors and configured to be stored at one ormore non-transitory computer readable media. Such non-transitorycomputer readable media can be part of system 300. The processor(s) canbe similar or identical to the processor(s) described above with respectto computer system 100 (FIG. 1).

In some embodiments, method 500 and other blocks in method 500 caninclude using a distributed network including distributed memoryarchitecture to perform the associated activity. This distributedarchitecture can reduce the impact on the network and system resourcesto reduce congestion in bottlenecks while still allowing data to beaccessible from a central location.

Referring to FIG. 5, method 500 can include a block 510 of receiving abasket for a user. In many embodiments, the basket (e.g., virtual cart)can include items that have been selected by the user, which can bereferred to as “basket items.” For example, a user can select a numberof items in an online grocery shopping system, and then choose toinitiate a checkout process. The user can be presented with a stock-uppage, which can include other items that are recommended for the user topurchase. In many embodiments, the items recommended on the stock-uppage can be determined using a basket-to-item model, as described belowin further detail.

In a number of embodiments, method 500 can continue with a block 520 ofsampling one item per category (e.g., L3 category). In many embodiments,each item in the basket can be grouped by the L3 category, which can bedenoted as L3 (1), L3 (2), . . . L3 (n−1), L3 (n), when there are n L3categories that are grouped from among the items in the basket. Inseveral embodiments, a single item can be randomly sampled from each ofthe L3 categories, and this randomly sampled item can be denoted as theanchor item for the L3 category. In some embodiments, if an item doesnot have an L3 category in its item taxonomy, then such item can befiltered out from the category grouping process.

In several embodiments, method 500 can continue with a group of blocks530 of generating complementary item recommendations, which can includeindividual instances for each of the L3 categories, such as blocks531-534. In many embodiments, each of blocks 531-534 can involvegenerating complementary item recommendations using the two sets oftrained item embeddings and trained user embeddings from the tripleembeddings model, as described above. For example, at block 531, theanchor item for category L3 (1) can be the given anchor item j describedabove, and the user can be the user u described above, such that thetrained embeddings can be used to generate a list of top k complementaryitem (i) recommendations, as described above. Blocks 532-534 cangenerate complementary item recommendations similarly for theirrespective anchor items. In many embodiments, each of blocks 531-534 canbe performed in parallel. In some embodiments, if no complementary itemrecommendations are generated for an anchor item in an L3 category, thena separate anchor item can be selected from that L3 category, such asanother random selection, and complementary item recommendations can begenerated for the newly selected anchor item in that L3 category.

In a number of embodiments, method 500 can continue with a group ofblocks 540 of performing a complementary category filtering, which caninclude individual instances for each of the L3 categories, such asblocks 541-544. In many embodiments, each of blocks 541-544 can involveperforming a complementary category filtering using the list ofcomplementary item recommendations generated in blocks 531-534,respectively. For example, at block 541, the list of complementary itemrecommendations generated at block 531 can be filtered to remove popularco-bought items having subcategories that are unrelated to thesubcategory of the anchor item, as described above. Blocks 542-544 canperform a complementary category filtering similarly for theirrespective lists of complementary item recommendations. In manyembodiments, each of blocks 541-544 can be performed in parallel. Eachrespective pair of blocks 531 and 541, blocks 532 and 542, blocks 533and 543, and blocks 534 and 544 can be an instance of using theitem-to-item model described above, which can generate lists ofcomplementary items for the anchor items randomly sampled in block 520.

In a number of embodiments, method 500 can continue with a block 550 ofperforming a weighted sampling. In many embodiments, for each of the L3categories, a respective quantity of items from the respective list ofthe complementary items can be sampled (e.g., randomly selected)proportional to a respective quantity of the basket items in therespective L3 category grouping with respect to a total quantity of thebasket items. In several embodiments, the number of recommended itemssampled for a L3 category can be calculated by multiplying the number ofbasket items in the L3 category by the number of total recommendationsto present to the user, divided by the total number of basket items inthe basket.

As an example, there can be 8 items in the basket that was selected by auser, such as 3 items in a first L3 category of “Canned Vegetables,” 3items in a second L3 category of “Pasta Sauce,” 1 item in a third L3category of “Broth, Stocks & Bouillon,” and 1 item in a fourth L3category of “Sports & Vitamin Drinks.” If the total number of itemrecommendations that will be generated for the basket is 40, then thenumber of item recommendations sampled for each of the first two L3categories can be 15, which is ⅜ of 40, and the number of itemrecommendations sampled for each of the last two L3 categories can be 5,which is ⅛ of 40. In another embodiment, the number k in the request forthe top k items requested in each of blocks 531-534 can be varied basedon the proportion of items in each L3 category.

In a number of embodiments, method 500 can continue with a block 560 ofoutputting a list of item recommendations for the basket. The itemrecommendations sampled in block 550 can be merged across all the L3categories (e.g., L3 (1) through L3 (n)). When merging, if the same itemis included in lists of item recommendations, the instance having thehighest score can be included in the merged list. In many embodiments,the list of item recommendations provided by the basket-to-item modelcan be personalized to the user based on items that are in the basket ina manner that considers more than just individual items in the basket.

Turning ahead in the drawings, FIG. 6 illustrates a flow chart for amethod 600, according to another embodiment. In some embodiments, method600 can be a method of providing personalized recommendations throughlarge-scale deep-embedding architecture. Method 600 is merely exemplaryand is not limited to the embodiments presented herein. Method 600 canbe employed in many different embodiments or examples not specificallydepicted or described herein. In some embodiments, the procedures, theprocesses, and/or the activities of method 600 can be performed in theorder presented. In other embodiments, the procedures, the processes,and/or the activities of method 600 can be performed in any suitableorder. In still other embodiments, one or more of the procedures, theprocesses, and/or the activities of method 600 can be combined orskipped. Method 600 can be similar to method 500 (FIG. 5), and variousactivities of method 600 can be similar or identical to variousactivities of method 500 (FIG. 5).

In many embodiments, system 300 (FIG. 3), personalized recommendationsystem 310 (FIG. 3), and/or web server 320 (FIG. 3) can be suitable toperform method 600 and/or one or more of the activities of method 600.In these or other embodiments, one or more of the activities of method600 can be implemented as one or more computing instructions configuredto run at one or more processors and configured to be stored at one ormore non-transitory computer readable media. Such non-transitorycomputer readable media can be part of system 300. The processor(s) canbe similar or identical to the processor(s) described above with respectto computer system 100 (FIG. 1).

In some embodiments, method 600 and other blocks in method 600 caninclude using a distributed network including distributed memoryarchitecture to perform the associated activity. This distributedarchitecture can reduce the impact on the network and system resourcesto reduce congestion in bottlenecks while still allowing data to beaccessible from a central location.

Referring to FIG. 6, method 600 optionally can include a block 605 oftraining a triple embeddings model with triplets using an adaptivemoment estimation optimizer to optimize a co-occurrence log-likelihoodof each of the triplets. In many embodiments, the triple embeddingsmodel can be similar or identical to the triple embeddings modeldescribed above, such as triple2vec. In some embodiments, the tripleembeddings model can be trained as describe above, based on pastpurchase history, using triplets of (user, first item, second item), inwhich the first item and the second item were selected (e.g., purchased)in the same basket by the user. In several embodiments, the training forthe triple embeddings model can be performed offline prior to using themodel. Once trained, the triple embeddings model can be used for aperiod of time to provide personalized item recommendations to manyusers. In a number of embodiments, the triple embeddings model can beretrained and/or updated periodically, such as weekly, or anothersuitable time period, to be updated with the latest purchase history.

In several embodiments, method 600 also can include a block 610 ofreceiving a basket comprising basket items selected by a user from anitem catalog. Block 610 can be similar to block 510 (FIG. 5). The usercan be similar or identical to user 350 (FIG. 3). As described above inconnection with block 510 (FIG. 5), the basket can include items thathave been selected by the user for purchase, referred to as “basketitems.” For example, the user can select a number of items in an onlinegrocery shopping system, and can initiate a checkout process.

In a number of embodiments, method 600 additionally can include a block615 of grouping the basket items of the basket into categories based ona respective item category of each of the basket items. In manyembodiments, the item categories can be one of the categorization levelsin an item taxonomy, such as the item taxonomy described above. Forexample, the item categories can be L3 categories, as described above.In various embodiments, each of the categories can be a group in whicheach of the items in the group has the same item category.

In several embodiments, method 600 further can include a block 620 ofrandomly sampling a respective anchor item from each of the categories.In many embodiments, the anchor item can be one of the basket items inthe category, chosen at random. For example, in a first category inwhich there are three basket items, the anchor item can be one of thethree basket items, randomly selected. The random sampling of anchoritems can be performed for each of the categories created in block 615.Blocks 615 and 620 can be similar to block 520 (FIG. 5).

In a number of embodiments, method 600 additionally can include a block625 of generating a respective list of complementary items for therespective anchor item for the each of the categories based on a scorefor each of the complementary items generated using two sets of traineditem embeddings for items in the item catalog and using trained userembeddings for the user. Block 625 can be similar to group of blocks 530(FIG. 5). The two set of trained item embeddings can be similar to theitem embeddings stored in matrices P and Q, described above. The traineduser embeddings for the user can be similar to h_(u) described above,which can be stored in matrix H, described above. The score can besimilar or identical to the cohesion score, described above. The itemsin the list of complementary items can be a top k items based on thegiven anchor item and the user, as described above.

In many embodiments, the two sets of trained item embeddings and theuser embeddings were trained using the triple embeddings model in block605 with triplets. The triplets can each include a respective first userof users, a respective first item from the item catalog, and arespective second item from the item catalog, in which the respectivefirst user selected the respective first item and the respective seconditem in a respective same basket. In a number of embodiments, a vectordimension for (i) the trained user embeddings for the user and (ii) foreach item in each of the two sets of trained item embeddings can be 128.In other embodiments, a different dimension can be used for theembedding vectors.

In several embodiments, method 600 further can include a block 630 ofbuilding a list of personalized recommended items for the user based onthe respective lists of the complementary items for the categories. Inmany embodiments, the lists of complementary items for the categoriesthat were generated in block 625 can be used to build the list ofpersonalized recommended items for the user.

In a number of embodiments, block 630 optionally can include a block 635of filtering the respective list of the complementary items for the eachof the categories based on complementary subcategories. Block 635 can besimilar to group of blocks 540 (FIG. 5). In several embodiments,filtering the respective list of complementary items can includereceiving lift scores for subcategories of the complementary item, andapplying the lift scores for the complementary items to remove, from therespective list of the complementary items, popular co-bought items inthe subcategories that are unrelated to a subcategory of the respectiveanchor item for the each of the categories. The subcategories can besimilar to the L4 subcategories described above. The lift scores can besimilar to the lift scores described above.

In several embodiments, block 630 also can include a block 640 ofperforming a weighted sampling of the respective list of thecomplementary items for the each of the categories to generate a sampledsub-list of the list of the complementary items for the each of thecategories. Block 640 can be similar to block 550 (FIG. 5). In severalembodiments, block 640 can include, for the each of the categories,sampling a respective quantity of items from the respective list of thecomplementary items proportional to a respective quantity of the basketitems in the each of the categories with respect to a total quantity ofthe basket items in the basket.

In a number of embodiments, block 630 additionally can include a block645 of merging the sampled sub-lists for the categories to generate aunified list. For example, the unified list can be a union of thesampled sub-lists created in block 640. In some embodiments, if the sameitem is in more than one of the sampled sub-lists, the item can beincluded once in the unified list, but the score of the item used can bethe highest score of that item based on the different cohesion scoresgenerated for that item across the different categories in which it wasrecommended for different anchor items.

In several embodiments, block 630 optionally can include a block 650 offiltering out items from the unified list, in which such filtered-outitems have subcategories that are identical to subcategories of thebasket items. For example, if a basket item in the basket has an L4subcategory of “Canned Corn,” then items in the unified list that havethat same L4 subcategory of “Canned Corn” can be removed from theunified list, so that the remaining items in the unified list will notbe too similar to what is already in the basket.

In a number of embodiments, block 630 further optionally can include ablock 655 of sorting each item in the unified list by the score of theitem. The score of the item can be the cohesion score determined foreach recommended item in block 625, which in some embodiments, wasadjusted based on the lift scores.

In several embodiments, block 630 further optionally can include a block660 of performing a category diversification across the unified list. Inmany embodiments, the unified list can be grouped into carousels ofrecommended items, such as groups of 5 recommended items. In a number ofembodiments, each carousel includes no more than one item of any itemcategory. For example, if there are multiple items in the unified listthat have item category of “Fabric Softeners,” only one of those itemswill be included in each carousel. In many embodiments, the carouselscan be presented in a round robin fashion. The category diversificationcan prevent recommending very similar items to the user at the sametime.

In a number of embodiments, method 600 additionally can include a block665 (after block 630) of sending instructions to display at least aportion of the list of personalized recommended items to the user. As anexample, the display can occur on a user interface of an electronicdevice. The list of personalized item recommendations can be the unifiedlist. In some embodiments, the entire list of personalized itemrecommendations can be displayed to the user, either all at once or inportions, such as in carousels that are presented round robin to displaythe entire list in segments. In other embodiments, only a portion of thelist of personalized item recommendations can be displayed to the user.For example, the top 10 items in the list can be displayed to the user,although the list of personalized item recommendations can include moreitems, such as 40 items. In many embodiments, the list of personalizeditem recommendations or portion thereof can be displayed on a stock-uppage that is presented to the user once the user initiates a checkoutprocess. The list of personalized item recommendations can include itemsthat are complementary to the items already in the basket andpersonalized to be compatible with the preferences of the user, aslearned through the triple embeddings model.

Turning ahead in the drawings, FIG. 7 illustrates a block diagram of asystem 700 that can be employed for providing personalizedrecommendations through large-scale deep-embedding architecture,according to another embodiment. System 700 is merely exemplary andembodiments of the system are not limited to the embodiments presentedherein. The system can be employed in many different embodiments orexamples not specifically depicted or described herein. In someembodiments, certain elements, modules, or systems of system 700 canperform various procedures, processes, and/or activities. In otherembodiments, the procedures, processes, and/or activities can beperformed by other suitable elements, modules, or systems of system 700.

Generally, therefore, system 700 can be implemented with hardware and/orsoftware, as described herein. In some embodiments, part or all of thehardware and/or software can be conventional, while in these or otherembodiments, part or all of the hardware and/or software can becustomized (e.g., optimized) for implementing part or all of thefunctionality of system 700 described herein.

System 700 can be similar to system 300 (FIG. 3), and various componentsof system 700 can be similar or identical to various components ofsystem 300 (FIG. 3). In many embodiments, system 700 can be animplementation of system 300 (FIG. 3) that is adapted with a real-timeinference method using approximate nearest neighbor (ANN) indexing. Inorder to provide personalized basket-to-item recommendations withlimited latencies, system 700 can be employed.

In conventional production item-item or user-item recommendationsystems, model recommendations are precomputed offline via batchcomputation, and cached in a database for static lookup in real-time.This approach cannot be applied to basket-to-item recommendations, dueto the exponential number of possible shopping baskets. Additionally,model inference time increases with basket size (e.g., number of itemsin the basket), which can increase latency.

The inference phase of the triple embeddings model (e.g., triple2vec)can be transformed into a similarity search of dense embedding vectors.For a given user u and anchor item j, this transformation can beachieved by adjusting the argmax of the cohesion score, as shown below:

${\underset{i}{argmax}\left( {{p_{i}^{T}q_{j}i} + {p_{i}^{T}h_{u}} + {q_{j}^{T}h_{u}}} \right)} = {\underset{i}{argmax}\left( {\begin{bmatrix}p_{i} & p_{i}\end{bmatrix}^{T}\begin{bmatrix}q_{j} & h_{u}\end{bmatrix}} \right)}$

The q_(j) ^(T)h_(u) term drops out of the argmax on the left side, as itis not based on i. The first term in the argmax on the right side,[p_(i) p_(i)]^(T), can be the ANN index, as it only depends on i. Thesecond term in the argmax on the right side, [q_(j) h_(u)], is based oninputs u and j, and can be the query vector. The argmax problem thus canbe transformed into a similarity search.

In some embodiments, another set of preference scores can be obtained byreversing p_(i) and q_(j) with q_(i) and p_(j), respectively. In manyembodiments, the model performance can be improved by interchanging thedual item embeddings and taking the average of the cohesion scores, asfollows:

${\underset{i}{argmax}\left( \frac{\left( {{p_{i}^{T}q_{j}} + {p_{i}^{T}h_{u}} + {q_{j}^{T}h_{u}}} \right) + \left( {{q_{i}^{T}p_{j}} + {q_{i}^{T}h_{u}} + {p_{j}^{T}h_{u}}} \right)}{2} \right)} = {{\underset{i}{argmax}\left( {{p_{i}^{T}q_{j}} + {p_{i}^{T}h_{u}} + {q_{i}^{T}p_{j}} + {q_{i}^{T}h_{u}}} \right)} = {\underset{i}{argmax}\left( {\begin{bmatrix}p_{i} & p_{i} & q_{i} & q_{i}\end{bmatrix}^{T}\begin{bmatrix}q_{j} & h_{u} & p_{j} & h_{u}\end{bmatrix}} \right)}}$

The first term in the argmax on the right side, [p_(i) p_(i) q_(i)q_(i)]^(T), is the ANN index, as it only depends on i. The second termin the argmax on the right side, [q_(j) h_(i), p_(j) h_(u)], is based oninputs u and j, and is the query vector.

In many embodiments, similarity search of the inference problem can besped up by using a conventional ANN indexing library, such as Faiss,Annoy, or NMSLIB (Non-Metric Space Library) to perform approximate dotproduct inference efficiently at scale.

In many embodiments, generating top-k within-basket recommendations inproduction can include: (1) basket-anchor set selection, (2) modelinference, and/or (3) post-processing. In some embodiments,basket-anchor set selection can include generating personalizedwithin-basket recommendations by replacing the item embeddings p_(i) andq_(i) with the average embedding of all the items in the shoppingbasket. This approach works very well for baskets with smaller sizes,but in practice, a typical family's shopping basket of groceriescontains dozens of items. Taking the average of such large basketsresults in losing information about the individual items in the basket.For larger baskets, a sampling algorithm that randomly selects 50% ofitems in the basket as a basket-anchor set can be used. In otherembodiments, a sampling approach similar or identical to the approachdescribed in block 520 (FIG. 5) and/or block 620 (FIG. 6) can be used toselect anchor items.

In several embodiments, model inference can include, for each item inthe basket-anchor set, creating the query vector [q_(j) h_(u) p_(j)h_(u)] using the pre-trained user embedding h_(u) and item embeddingsp_(i) and q_(i). The query vector can be used in the ANN index toretrieve the top-k recommendations. The ANN index can be created fromthe concatenation of the dual item embeddings [p_(i) p_(i) q_(i) q_(i)]for all i. The ANN index and embeddings can be stored in memory for fastlookup. In practice, the inference can be further sped up by performinga batch lookup in the ANN index instead of performing a sequentiallookup for each item in the basket-anchor set. In many embodiments, kcan be set to 30, such that the top 30 nearest neighbors. In otherembodiments, another suitable value can be pre-selected, or another thevalue can be set variably customized based on one or more factors.

After the top-k recommendations are retrieved for each anchor item inthe basket-anchor set, a recommendation aggregator system can be used toblend all the recommendations together. The aggregator can use severalfactors such as number of distinct categories in the recommendation set,the individual item scores in the recommendations, taxonomy-basedweighting, and business rules to merge the multiple recommendation sets,and filter to a top-k recommendation set. Once the top-k recommendationset is generated, an additional post-processing layer can be applied.This layer can incorporate diversification of items, remove blacklisteditems and categories, utilize market-basket analysis association rulesfor taxonomy-based filtering, and/or apply various business requirementsto generate the final top-k recommendations for production serving.

As shown in FIG. 7, system 700 can be used to implement thiswithin-basket real-time recommendation system using ANN embeddingretrieval. System 700 can include online components 710 and offlinecomponents 720. Online components 710 can include a distributedstreaming engine 711, a front-end client 712, a real-time inferenceengine 713, and/or an embedding lookup cache 714. Offline components 720can include a data store 721, a task engine 722, a feature store 723, anoffline deep-learning model 724, user embeddings 725, trained model 726,and/or cache data loader script 727.

In many embodiments, streaming engine 711 can handle the transactionsdata as they are received across the system from the users. For example,a Kafka streaming engine can be used to capture real-time customer datain real-time and store the data in a data store 721, such as aHadoop-based distributed file system. For offline model training, taskengine 722 can construct training examples by extracting features fromfeature store 723, such as through using Hive or Spark jobs. Thetraining examples can be input into offline deep learning model 724,which can be trained offline on a GPU cluster, for example, to generateuser embeddings 725 and dual-item embeddings, which can be used toconstruct an ANN index in trained model 726. User embeddings 725 can bestored by cache data script loader 727 in embedding lookup cache 714,such as a distributed cache, to facilitate online retrieval by real-timeinference engine 713. For example, real-time inference engine 713 cancall embedding lookup cache 714 using a user identifier to obtain theuser embedding for the user and/or the query vector for the user.

In many embodiments, real-time inference engine 713 can providepersonalized recommendations, while providing high throughput and alow-latency experience to the user. In several embodiments, real-timeinference engine 713 can utilize the ANN index in trained model 726,constructed from the trained embeddings, and deployed as amicro-service. In a number of embodiments, real-time inference engine713 can interact with front-end client 712, which can be similar to webserver 320 (FIG. 3) to obtain user and basket context and generatespersonalized within-basket recommendations in real-time. In someembodiments, the offline training can be performed periodically, such asweekly, or at another suitable interval, to handle new past-purchasetransaction data to update the model.

The model described above for system 700 was evaluated to determinelatency performance with various ANN indexing libraries. For parametersettings in the model, an embedding size of 64 was used, along with theAdam Optimizer with an initial learning rate of 1.0, and thenoise-contrastive estimation (NCE) of softmax as the loss function. Abatch size of 1000 and a maximum of 100 epochs was used to train themodel. 200 million triplets were used to train the dataset.

The real-time latency of system 700 was tested using exact inference andapproximate inference methods as described above. Turning ahead in thedrawings, FIG. 8 illustrates a graph 800 showing inference latency (inmilliseconds (ms)) versus basket size (in number of basket items). ND4Jwas used to perform exact inference based the following argmax approach:

$\underset{i}{argmax}\left( {\begin{bmatrix}p_{i} & p_{i} & q_{i} & q_{i}\end{bmatrix}^{T}\begin{bmatrix}q_{j} & h_{u} & p_{j} & h_{u}\end{bmatrix}} \right)$

Approximate inferencing also was tested using the Faiss, Annoy, andNMSLIB libraries. ND4J is a highly-optimized scientific computinglibrary for the Java Virtual Machine (JVM). Faiss is used for efficientsimilarity search of dense vectors that can scale to billions ofembeddings. Annoy is an ANN library optimized for memory usage andloading/saving to disk. NMSLIB is a similarity search library forgeneric nonmetric spaces.

As shown in FIG. 8, on average, ND4J adds 186.5 ms of latency whenperforming exact real-time inference. For approximate inference, Faiss,Annoy, and NMSLIB libraries add an additional 29.3 ms, 538.7 ms, and16.07 ms of system latency respectively. Faiss and NMSLIB provide anoption to perform batch queries on the index, therefore latency is muchlower than Annoy. Faiss and NMSLIB are 6-10 times faster than the exactinference method using ND4J. In many embodiments, the real-timeuser-personalized within-basket recommendation system can servepersonalized item recommendations at large-scale with low latency.

Turning ahead in the drawings, FIG. 9 illustrates a flow chart for amethod 900, according to another embodiment. In some embodiments, method900 can be a method of providing personalized recommendations throughlarge-scale deep-embedding architecture. Method 900 is merely exemplaryand is not limited to the embodiments presented herein. Method 900 canbe employed in many different embodiments or examples not specificallydepicted or described herein. In some embodiments, the procedures, theprocesses, and/or the activities of method 900 can be performed in theorder presented. In other embodiments, the procedures, the processes,and/or the activities of method 900 can be performed in any suitableorder. In still other embodiments, one or more of the procedures, theprocesses, and/or the activities of method 900 can be combined orskipped. Method 900 can be similar to method 500 (FIG. 5) and/or method600 (FIG. 6), and various activities of method 900 can be similar oridentical to various activities of method 500 (FIG. 5) and/or method 600(FIG. 6).

In many embodiments, system 300 (FIG. 3), personalized recommendationsystem 310 (FIG. 3), and/or web server 320 (FIG. 3) can be suitable toperform method 900 and/or one or more of the activities of method 900.In these or other embodiments, one or more of the activities of method900 can be implemented as one or more computing instructions configuredto run at one or more processors and configured to be stored at one ormore non-transitory computer readable media. Such non-transitorycomputer readable media can be part of system 300. The processor(s) canbe similar or identical to the processor(s) described above with respectto computer system 100 (FIG. 1).

In some embodiments, method 900 and other blocks in method 900 caninclude using a distributed network including distributed memoryarchitecture to perform the associated activity. This distributedarchitecture can reduce the impact on the network and system resourcesto reduce congestion in bottlenecks while still allowing data to beaccessible from a central location.

Referring to FIG. 9, method 900 optionally can include a block 905 oftraining two sets of item embeddings for items in an item catalog and aset of user embeddings for users, using a triple embeddings model, withtriplets. In many embodiments, the triple embeddings model can besimilar or identical to the triple embeddings model described above,such as triple2vec. In some embodiments, the triple embeddings model canbe trained as describe above, based on past purchase history, usingtriplets. In several embodiments, the triplets each can include arespective first user of the users, a respective first item from theitem catalog, and a respective second item from the item catalog, inwhich the respective first user selected the respective first item andthe respective second item in a respective same basket. In other words,each triplet can include (user, first item, second item), in which thefirst item and the second item were selected (e.g., purchased) in thesame basket by the user. Block 905 can be similar or identical to block605 (FIG. 6). The two set of trained item embeddings can be similar tothe item embeddings stored in matrices P and Q, described above. Thetrained user embeddings for the user can be similar to h_(u) describedabove, which can be stored in matrix H, described above. In a number ofembodiments, a vector dimension for (i) the trained user embeddings forthe user and (ii) for each item in each of the two sets of trained itemembeddings can be 128. In other embodiments, a different dimension canbe used for the embedding vectors.

In a number of embodiments, method 900 additionally can include a block910 of generating an approximate nearest neighbor (ANN) index for thetwo sets of item embeddings. In many embodiments, the ANN index can besimilar or identical to the ANN index described above. In variousembodiments, the ANN index can be generated and implemented using aconventional similarity search library and/or ANN indexing library, suchas Faiss, Annoy, or NMSLIB, as described above. In a number ofembodiments, the triple embeddings model and/or the approximate nearestneighbor index can be periodically precomputed using the ANN indexlibrary and/or similarity search library. In many embodiments, thetraining in block 905 and generating the ANN index can be performed inoffline deep-learning model 724 (FIG. 7), as described above. Thetrained model with ANN index can be deployed in real-time inferenceengine 713 (FIG. 7), and the user embeddings can be deployed inembedding lookup cache 714 (FIG. 7). In many embodiments, the set ofuser embeddings for the users are loaded into a memory cache, such asembedding lookup cache 714 (FIG. 7), before the respective lookup callsare made.

In several embodiments, method 900 also further include a block 915 ofreceiving a basket comprising basket items selected by a user from theitem catalog. Block 915 can be similar or identical to block 510 (FIG.5) and/or block 610 (FIG. 6). The user can be similar or identical touser 350 (FIG. 3). As described above, the basket can include items thathave been selected by the user for purchase, referred to as “basketitems.” For example, the user can select a number of items in an onlinegrocery shopping system, and can initiate a checkout process. In manyembodiments, the basket can be received in block 915, such as from acall to real-time inference engine 713 (FIG. 7) from front-end client712 (FIG. 7) that includes information about the user and the basket.

In a number of embodiments, method 900 additionally can include a block920 of grouping the basket items of the basket into categories based ona respective item category of each of the basket items. In manyembodiments, the item categories can be one of the categorization levelsin an item taxonomy, such as the item taxonomy described above. Forexample, the item categories can be L3 categories, as described above.In various embodiments, each of the categories can be a group in whicheach of the items in the group has the same item category. Block 920 canbe similar or identical to block 615 (FIG. 6).

In several embodiments, method 900 further can include a block 925 ofrandomly sampling a respective anchor item from each of the categories.In many embodiments, the anchor item can be one of the basket items inthe category, chosen at random. For example, in a first category inwhich there are three basket items, the anchor item can be one of thethree basket items, randomly selected. The random sampling of anchoritems can be performed for each of the categories created in block 915.Block 925 can be similar or identical to block 620 (FIG. 6). Blocks 920and 925 together can be similar to block 520 (FIG. 5).

In a number of embodiments, method 900 additionally can include a block930 of generating a respective list of complementary items for therespective anchor item for the each of the categories based on arespective lookup call to the approximate nearest neighbor index using aquery vector associated with the user and the respective anchor item.Block 925 can be similar to group of blocks 530 (FIG. 5) and/or block625 (FIG. 6), but can involve an approximate inferencing approach usingthe ANN index. The items in the list of complementary items can be a topk items based on the given anchor item and the user, as described above.The query vector can be similar or identical to the query vectordescribed above. In many embodiments, the query vector can be generatedfor the user and the respective anchor item using the two sets of itemembeddings and the set of user embeddings. In many embodiments, the listof complementary items can be generated in block 930 for each categoryusing real-time inference engine 713 (FIG. 7), as described above. In anumber of embodiments, the respective lookup calls to the approximatelynearest neighbor index can be made in parallel across the categories,similarly as shown in group of blocks 530 (FIG. 5) and described above.

In several embodiments, method 900 further can include a block 935 ofbuilding a list of personalized recommended items for the user based onthe respective lists of the complementary items for the categories. Inmany embodiments, the lists of complementary items for the categoriesthat were generated in block 930 can be used to build the list ofpersonalized recommended items for the user. In several embodiments,block 935 can be similar or identical to block 630 (FIG. 6), and caninclude one or more of blocks 635, 640, 645, 650, 655, and/or 660 (FIG.6).

In a number of embodiments, method 900 additionally can include a block940 of sending instructions to display at least a portion of the list ofpersonalized recommended items to the user. As an example, the displaycan occur on a user interface of an electronic device. In someembodiments, the entire list of personalized recommended items can bedisplayed to the user, either all at once or in portions, such as incarousels that are presented round robin to display the entire list insegments. In other embodiments, only a portion of the list ofpersonalized recommended items can be displayed to the user. Forexample, the top 10 items in the list can be displayed to the user,although the list can include more items, such as 40 items. In manyembodiments, the list of personalized recommended items or portionthereof can be displayed on a stock-up page that is presented to theuser once the user initiates a checkout process. The list ofpersonalized recommended items can include items that are complementaryto the items already in the basket and personalized to be compatiblewith the preferences of the user, as learned through the tripleembeddings model.

Returning to FIG. 3, in several embodiments, communication system 311can at least partially perform block 510 (FIG. 5) of receiving a basketfor a user; block 560 (FIG. 5) of outputting a list of itemrecommendations for the basket; block 610 (FIG. 6) of receiving a basketcomprising basket items selected by a user from an item catalog; and/orblock 915 (FIG. 9) of receiving a basket comprising basket itemsselected by a user from the item catalog.

In several embodiments, item-to-item system 312 can at least partiallyperform group of blocks 530 (FIG. 5) of generating complementary itemrecommendations; block 625 (FIG. 6) of generating a respective list ofcomplementary items for the respective anchor item for the each of thecategories based on a score for each of the complementary itemsgenerated using two sets of trained item embeddings for items in theitem catalog and using trained user embeddings for the user; and/orblock 930 (FIG. 9) of generating a respective list of complementaryitems for the respective anchor item for the each of the categoriesbased on a respective lookup call to the approximate nearest neighborindex using a query vector associated with the user and the respectiveanchor item.

In a number of embodiments, basket-to-item system 313 can at leastpartially perform block 520 (FIG. 5) of sampling one item per category;block 615 (FIG. 6) of grouping the basket items of the basket intocategories based on a respective item category of each of the basketitems; block 620 (FIG. 6) of randomly sampling a respective anchor itemfrom each of the categories; block 920 (FIG. 9) of grouping the basketitems of the basket into categories based on a respective item categoryof each of the basket items; and/or block 925 (FIG. 9) of randomlysampling a respective anchor item from each of the categories.

In several embodiments, triple embeddings system 314 can at leastpartially perform block 605 (FIG. 6) of training a triple embeddingsmodel with triplets using an adaptive moment estimation optimizer tooptimize a co-occurrence log-likelihood of each of the triplets; and/orblock 905 (FIG. 9) of training two sets of item embeddings for items inan item catalog and a set of user embeddings for users, using a tripleembeddings model, with triplets.

In several embodiments, post-processing system 315 can at leastpartially perform group of blocks 540 (FIG. 5) of performing acomplementary category filtering; block 550 (FIG. 5) of performing aweighted sampling; block 630 (FIG. 6) of building a list of personalizedrecommended items for the user based on the respective lists of thecomplementary items for the categories; block 635 (FIG. 6) of filteringthe respective list of the complementary items for the each of thecategories based on complementary subcategories; block 640 (FIG. 6) ofperforming a weighted sampling of the respective list of thecomplementary items for the each of the categories to generate a sampledsub-list of the list of the complementary items for the each of thecategories; block 645 (FIG. 6) of merging the sampled sub-lists for thecategories to generate a unified list; block 650 (FIG. 6) of filteringout items from the unified list having subcategories that are identicalto subcategories of the basket items; block 655 (FIG. 6) of sorting eachitem in the unified list by the score of the item; block 660 (FIG. 6) ofperforming a category diversification across the unified list; and/orblock 935 (FIG. 9) of building a list of personalized recommended itemsfor the user based on the respective lists of the complementary itemsfor the categories.

In a number of embodiments, ANN index system 316 can at least partiallyperform block 910 (FIG. 9) of generating an approximate nearest neighbor(ANN) index for the two sets of item embeddings; and/or block 930 (FIG.9) of generating a respective list of complementary items for therespective anchor item for the each of the categories based on arespective lookup call to the approximate nearest neighbor index using aquery vector associated with the user and the respective anchor item.

In a number of embodiments, web server 320 can at least partiallyperform block 510 (FIG. 5) of receiving a basket for a user; block 560(FIG. 5) of outputting a list of item recommendations for the basket;block 610 (FIG. 6) of receiving a basket comprising basket itemsselected by a user from an item catalog; block 665 (FIG. 6) of sendinginstructions to display at least a portion of the list of personalizedrecommended items to the user; and/or block 940 (FIG. 9) of sendinginstructions to display at least a portion of the list of personalizedrecommended items to the user.

In many embodiments, the techniques described herein can provide apractical application and several technological improvements.Specifically, the techniques described herein can provide forautomatically providing personalized recommendations through large-scaledeep-embedding architecture, which can provide more relevant itemrecommendations that are compatible with the preferences of the user andcomplementary with the items in the basket. In a number of embodiments,the techniques described herein can use a novel machine-learningapproach that can learn features that can represent complementaritybetween items and/or compatibility between users and items, which can beused to provide more relevant personalized recommendations for a userbased on the items that the user has selected in the basket. In manyembodiments, this model can be implemented with an approximate inferencetechnique to lower the latency and provide the item recommendations inreal-time, which can be scaled to serve millions of online users. In anumber of embodiments, the techniques described herein can solve atechnical problem that cannot be solved outside the context of computernetworks. For example, the machine learning models described here cannotbe implemented outside the context of computer networks.

Various embodiments can include a system including one or moreprocessors and one or more non-transitory computer-readable mediastoring computing instructions configured to run on the one or moreprocessors and perform certain acts. The acts can include receiving abasket including basket items selected by a user from an item catalog.The acts also can include grouping the basket items of the basket intocategories based on a respective item category of each of the basketitems. The acts additionally can include randomly sampling a respectiveanchor item from each of the categories. The acts further can includegenerating a respective list of complementary items for the respectiveanchor item for the each of the categories based on a respective scorefor each of the complementary items generated using two sets of traineditem embeddings for items in the item catalog and using trained userembeddings for the user. The two sets of trained item embeddings and thetrained user embeddings can be trained using a triple embeddings modelwith triplets. The triplets each can include a respective first user ofusers, a respective first item from the item catalog, and a respectivesecond item from the item catalog, in which the respective first userselected the respective first item and the respective second item in arespective same basket. The acts additionally can include building alist of personalized recommended items for the user based on therespective lists of the complementary items for the categories. The actsfurther can include sending instructions to display, to the user on auser interface of a user device, at least a portion of the list ofpersonalized recommended items.

A number of embodiments can include a method being implemented viaexecution of computing instructions configured to run at one or moreprocessors and stored at one or more non-transitory computer-readablemedia. The method can include receiving a basket including basket itemsselected by a user from an item catalog. The method also can includegrouping the basket items of the basket into categories based on arespective item category of each of the basket items. The methodadditionally can include randomly sampling a respective anchor item fromeach of the categories. The method further can include generating arespective list of complementary items for the respective anchor itemfor the each of the categories based on a respective score for each ofthe complementary items generated using two sets of trained itemembeddings for items in the item catalog and using trained userembeddings for the user. The two sets of trained item embeddings and thetrained user embeddings can be trained using a triple embeddings modelwith triplets. The triplets each can include a respective first user ofusers, a respective first item from the item catalog, and a respectivesecond item from the item catalog, in which the respective first userselected the respective first item and the respective second item in arespective same basket. The method additionally can include building alist of personalized recommended items for the user based on therespective lists of the complementary items for the categories. Themethod further can include sending instructions to display, to the useron a user interface of a user device, at least a portion of the list ofpersonalized recommended items.

Various embodiments can include a system including one or moreprocessors and one or more non-transitory computer-readable mediastoring computing instructions configured to run on the one or moreprocessors and perform certain acts. The acts can include training twosets of item embeddings for items in an item catalog and a set of userembeddings for users, using a triple embeddings model, with triplets.The triplets each can include a respective first user of the users, arespective first item from the item catalog, and a respective seconditem from the item catalog, in which the respective first user selectedthe respective first item and the respective second item in a respectivesame basket. The acts also can include generating an approximate nearestneighbor index for the two sets of item embeddings. The actsadditionally can include receiving a basket including basket itemsselected by a user from the item catalog. The acts further can includegrouping the basket items of the basket into categories based on arespective item category of each of the basket items. The actsadditionally can include randomly sampling a respective anchor item fromeach of the categories. The acts further can include generating arespective list of complementary items for the respective anchor itemfor the each of the categories based on a respective lookup call to theapproximate nearest neighbor index using a query vector associated withthe user and the respective anchor item. The acts additionally caninclude building a list of personalized recommended items for the userbased on the respective lists of the complementary items for thecategories. The acts further can include sending instructions todisplay, to the user on a user interface of a user device, at least aportion of the list of personalized recommended items.

A number of embodiments can include a method being implemented viaexecution of computing instructions configured to run at one or moreprocessors and stored at one or more non-transitory computer-readablemedia. The method can include training two sets of item embeddings foritems in an item catalog and a set of user embeddings for users, using atriple embeddings model, with triplets. The triplets each can include arespective first user of the users, a respective first item from theitem catalog, and a respective second item from the item catalog, inwhich the respective first user selected the respective first item andthe respective second item in a respective same basket. The method alsocan include generating an approximate nearest neighbor index for the twosets of item embeddings. The method additionally can include receiving abasket including basket items selected by a user from the item catalog.The method further can include grouping the basket items of the basketinto categories based on a respective item category of each of thebasket items. The method additionally can include randomly sampling arespective anchor item from each of the categories. The method furthercan include generating a respective list of complementary items for therespective anchor item for the each of the categories based on arespective lookup call to the approximate nearest neighbor index using aquery vector associated with the user and the respective anchor item.The method additionally can include building a list of personalizedrecommended items for the user based on the respective lists of thecomplementary items for the categories. The method further can includesending instructions to display, to the user on a user interface of auser device, at least a portion of the list of personalized recommendeditems.

Although the methods described above are with reference to theillustrated flowcharts, it will be appreciated that many other ways ofperforming the acts associated with the methods can be used. Forexample, the order of some operations may be changed, and some of theoperations described may be optional.

In addition, the methods and system described herein can be at leastpartially embodied in the form of computer-implemented processes andapparatus for practicing those processes. The disclosed methods may alsobe at least partially embodied in the form of tangible, non-transitorymachine-readable storage media encoded with computer program code. Forexample, the steps of the methods can be embodied in hardware, inexecutable instructions executed by a processor (e.g., software), or acombination of the two. The media may include, for example, RAMs, ROMs,CD-ROMs, DVD-ROMs, BD-ROMs, hard disk drives, flash memories, or anyother non-transitory machine-readable storage medium. When the computerprogram code is loaded into and executed by a computer, the computerbecomes an apparatus for practicing the method. The methods may also beat least partially embodied in the form of a computer into whichcomputer program code is loaded or executed, such that, the computerbecomes a special purpose computer for practicing the methods. Whenimplemented on a general-purpose processor, the computer program codesegments configure the processor to create specific logic circuits. Themethods may alternatively be at least partially embodied in applicationspecific integrated circuits for performing the methods.

The foregoing is provided for purposes of illustrating, explaining, anddescribing embodiments of these disclosures. Modifications andadaptations to these embodiments will be apparent to those skilled inthe art and may be made without departing from the scope or spirit ofthese disclosures.

Although providing personalized recommendations through large-scaledeep-embedding architecture has been described with reference tospecific embodiments, it will be understood by those skilled in the artthat various changes may be made without departing from the spirit orscope of the disclosure. Accordingly, the disclosure of embodiments isintended to be illustrative of the scope of the disclosure and is notintended to be limiting. It is intended that the scope of the disclosureshall be limited only to the extent required by the appended claims. Forexample, to one of ordinary skill in the art, it will be readilyapparent that any element of FIGS. 1-9 may be modified, and that theforegoing discussion of certain of these embodiments does notnecessarily represent a complete description of all possibleembodiments. For example, one or more of the procedures, processes, oractivities of FIG. 5, 6 or 9 may include different procedures,processes, and/or activities and be performed by many different modules,in many different orders. As another example, one or more of theprocedures, processes, and/or activities of one of FIG. 5, 6 or 9 can beperformed in another one of FIG. 5, 6 or 9. As another example, thesystems within system 300 in FIG. 3 can be interchanged or otherwisemodified.

Replacement of one or more claimed elements constitutes reconstructionand not repair. Additionally, benefits, other advantages, and solutionsto problems have been described with regard to specific embodiments. Thebenefits, advantages, solutions to problems, and any element or elementsthat may cause any benefit, advantage, or solution to occur or becomemore pronounced, however, are not to be construed as critical, required,or essential features or elements of any or all of the claims, unlesssuch benefits, advantages, solutions, or elements are stated in suchclaim.

Moreover, embodiments and limitations disclosed herein are not dedicatedto the public under the doctrine of dedication if the embodiments and/orlimitations: (1) are not expressly claimed in the claims; and (2) are orare potentially equivalents of express elements and/or limitations inthe claims under the doctrine of equivalents.

What is claimed is:
 1. A system comprising: one or more processors; andone or more non-transitory computer-readable media storing computinginstructions configured to run on the one or more processors andperform: training two sets of item embeddings for items in an itemcatalog and a set of user embeddings for users, using a tripleembeddings model, with triplets, wherein the triplets each comprise arespective first user of the users, a respective first item from theitem catalog, and a respective second item from the item catalog, inwhich the respective first user selected the respective first item andthe respective second item in a respective same basket; generating anapproximate nearest neighbor index for the two sets of item embeddings;receiving a basket comprising basket items selected by a user from theitem catalog; grouping the basket items of the basket into categoriesbased on a respective item category of each of the basket items;randomly sampling a respective anchor item from each of the categories;generating a respective list of complementary items for the respectiveanchor item for the each of the categories based on a respective lookupcall to the approximate nearest neighbor index using a query vectorassociated with the user and the respective anchor item; building a listof personalized recommended items for the user based on the respectivelists of the complementary items for the categories; and sendinginstructions to display, to the user on a user interface of a userdevice, at least a portion of the list of personalized recommendeditems.
 2. The system of claim 1, wherein the query vector is generatedfor the user and the respective anchor item using the two sets of itemembeddings and the set of user embeddings.
 3. The system of claim 1,wherein the approximate nearest neighbor index is periodicallyprecomputed using a similarity search library.
 4. The system of claim 1,wherein the set of user embeddings for the users are loaded into amemory cache before the respective lookup calls are made.
 5. The systemof claim 1, wherein the respective lookup calls to the approximatelynearest neighbor index are made in parallel across the categories. 6.The system of claim 1, wherein building the list of personalizedrecommended items for the user further comprises: filtering therespective list of the complementary items for the each of thecategories based on complementary subcategories; performing a weightedsampling of the respective list of the complementary items for the eachof the categories to generate a sampled sub-list of the list of thecomplementary items for the each of the categories; and merging thesampled sub-lists for the categories to generate a unified list.
 7. Thesystem of claim 6, wherein filtering the respective list of thecomplementary items for the each of the categories based on thecomplementary subcategories further comprises: receiving lift scores forsubcategories of the complementary items; and applying the lift scoresfor the complementary items to remove, from the respective list of thecomplementary items, popular co-bought items in the subcategories thatare unrelated to a subcategory of the respective anchor item for theeach of the categories.
 8. The system of claim 6, wherein performing theweighted sampling of the respective list of the complementary items forthe each of the categories further comprises: for the each of thecategories, sampling a respective quantity of items from the respectivelist of the complementary items proportional to a respective quantity ofthe basket items in the each of the categories with respect to a totalquantity of the basket items in the basket.
 9. The system of claim 1,wherein the triple embeddings model is trained with the triplets usingan adaptive moment estimation optimizer to optimize a co-occurrencelog-likelihood of each of the triplets.
 10. The system of claim 1,wherein the portion of the list of personalized recommended items isdisplayed to the user on a checkout page for the basket, wherein thecheckout page appears on the user interface of the user device.
 11. Amethod being implemented via execution of computing instructionsconfigured to run at one or more processors and stored at one or morenon-transitory computer-readable media, the method comprising: trainingtwo sets of item embeddings for items in an item catalog and a set ofuser embeddings for users, using a triple embeddings model, withtriplets, wherein the triplets each comprise a respective first user ofthe users, a respective first item from the item catalog, and arespective second item from the item catalog, in which the respectivefirst user selected the respective first item and the respective seconditem in a respective same basket; generating an approximate nearestneighbor index for the two sets of item embeddings; receiving a basketcomprising basket items selected by a user from the item catalog;grouping the basket items of the basket into categories based on arespective item category of each of the basket items; randomly samplinga respective anchor item from each of the categories; generating arespective list of complementary items for the respective anchor itemfor the each of the categories based on a respective lookup call to theapproximate nearest neighbor index using a query vector associated withthe user and the respective anchor item; building a list of personalizedrecommended items for the user based on the respective lists of thecomplementary items for the categories; and sending instructions todisplay, to the user on a user interface of a user device, at least aportion of the list of personalized recommended items.
 12. The method ofclaim 11, wherein the query vector is generated for the user and therespective anchor item using the two sets of item embeddings and the setof user embeddings.
 13. The method of claim 11, wherein the approximatenearest neighbor index is periodically precomputed using a similaritysearch library.
 14. The method of claim 11, wherein the set of userembeddings for the users are loaded into a memory cache before therespective lookup calls are made.
 15. The method of claim 11, whereinthe respective lookup calls to the approximately nearest neighbor indexare made in parallel across the categories.
 16. The method of claim 11,wherein building the list of personalized recommended items for the userfurther comprises: filtering the respective list of the complementaryitems for the each of the categories based on complementarysubcategories; performing a weighted sampling of the respective list ofthe complementary items for the each of the categories to generate asampled sub-list of the list of the complementary items for the each ofthe categories; and merging the sampled sub-lists for the categories togenerate a unified list.
 17. The method of claim 16, wherein filteringthe respective list of the complementary items for the each of thecategories based on the complementary subcategories further comprises:receiving lift scores for subcategories of the complementary items; andapplying the lift scores for the complementary items to remove, from therespective list of the complementary items, popular co-bought items inthe subcategories that are unrelated to a subcategory of the respectiveanchor item for the each of the categories.
 18. The method of claim 16,wherein performing the weighted sampling of the respective list of thecomplementary items for the each of the categories further comprises:for the each of the categories, sampling a respective quantity of itemsfrom the respective list of the complementary items proportional to arespective quantity of the basket items in the each of the categorieswith respect to a total quantity of the basket items in the basket. 19.The method of claim 11, wherein the triple embeddings model is trainedwith the triplets using an adaptive moment estimation optimizer tooptimize a co-occurrence log-likelihood of each of the triplets.
 20. Themethod of claim 11, wherein the portion of the list of personalizedrecommended items is displayed to the user on a checkout page for thebasket, wherein the checkout page appears on the user interface of theuser device.